Image capturing apparatus, an image capturing method and a machine readable medium storing thereon a computer program for capturing an image of a range wider than an image capture designation range

ABSTRACT

It is an object to provide an image of a desired composition in case the user is unsatisfied with the composition of an image already captured. 
     An image capturing apparatus for capturing an image is provided, wherein the apparatus includes an image capturing designation range acquiring unit for acquiring an image capturing designation range designated by a user; an image capturing unit for capturing an image of a range wider than the image capturing designation range; a storing unit for storing information indicative of the image capturing designation range corresponding to the captured image; and an image displaying unit for displaying at least an image within the image capturing designation range out of the captured image by making the user recognize the image capturing designation range.

CROSS REFERENCE TO THE RELATED APPLICATION

The present application is a Divisional of U.S. application Ser. No.11/220,613, filed on Sep. 8, 2005, which claims priority from a JapanesePatent Application No. JP 2005-184412 filed on Jun. 24, 2005, thecontents of which are incorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to an image capturing apparatus, an imagecapturing method and a machine readable medium storing thereon acomputer program for capturing an image. More particularly, the presentinvention relates to an image capturing apparatus for capturing animage, an image capturing method for capturing an image and a machinereadable medium storing thereon a computer program for capturing animage.

DESCRIPTION OF THE RELATED ART

Conventionally, a digital camera records data of a captured digitalimage on a recording medium using a nonvolatile memory, such as a flashmemory. Recently, as the recording capacity of the recording medium usedin the digital camera is increased, a user can record more images orhigher quality images than before.

Thus, it is disclosed that an image capturing apparatus which canprovide better quality image even after the image has been captured atthe quality level designated by the user when the user wants to enhancethe quality of captured image because he/she thinks the image was verywell composed as disclosed, for example, in Japanese Patent ApplicationLaid-open No. 2000-295577.

SUMMARY OF THE INVENTION

It is also frequent for a general user to be unsatisfied with thecomposition of a captured image, saying “it should have been better toshift the image to the right a little bit . . . ”, because it is not soeasy for her to take a good composition every time when she captures animage. The conventional image capturing apparatus disclosed in the aboveJapanese patent application may provide a solution to the user who isunsatisfied with the poor quality of the well-composed image, but doesnot provide any solution to a user who is unsatisfied with thecomposition of the captured image.

Therefore, it is an object of the present invention to provide an imagecapturing apparatus, an image capturing method and a machine readablemedium storing thereon a computer program for capturing an image, whichis capable of overcoming the above drawbacks accompanying theconventional art. The above and other objects can be achieved bycombinations described in the independent claims. The dependent claimsdefine further advantageous and exemplary combinations of the presentinvention.

According to the first aspect of the present invention, an imagecapturing apparatus for capturing an image includes: an image capturingdesignation range acquiring unit for acquiring an image capturingdesignation range designated by a user; an image capturing unit forcapturing an image of a range wider than the image capturing designationrange; a storing unit for storing information indicative of the imagecapturing designation range corresponding to the captured image; and animage displaying unit for displaying at least an image within the imagecapturing designation range out of the captured image by making the userrecognize the image capturing designation range.

The image capturing apparatus may further include: an image processingunit for performing an image processing on the captured image in orderto make an amount of data of the portion of the image outside of theimage capturing designation range be less than that of the portion ofthe image within the image capturing designation range out of thecaptured image. The image processing unit may convert the portion of theimage outside of the image capturing designation range into an image ofa resolution lower than that of the portion of the image within theimage capturing designation range out of the captured image. The imageprocessing unit may compress the portion of the image outside of theimage capturing designation range at a compression rate higher than thatfor the portion of the image within the image capturing designationrange out of the captured image. Further, the image processing unit mayperform an image enhancing process on the portion of the image withinthe image capturing designation range at a level higher than that forthe portion of the image outside of the image capturing designationrange out of the captured image. Furthermore, the image capturingapparatus may capture a moving picture, the image capturing unit maycapture an image wider than the image capturing designation range as thecaptured image which is a frame of the moving picture, and the imageprocessing unit may remove more frequency components from image signalsof the portion of the image outside the image capturing designationrange than those of the portion of the image within the image capturingdesignation range.

The image capturing apparatus may further include: a remaining capacitydetecting unit for detecting remaining capacity of the storing unit; andan image removing unit for removing a portion of the image outside ofthe image capturing designation range corresponding to that capturedimage stored in the storing unit in case the remaining capacity detectedby the remaining capacity detecting unit. The image capturing apparatusmay further include: a remaining capacity displaying unit for displayinga capacity calculated by adding the remaining capacity detected by theremaining capacity detecting unit to the capacity occupied by theportion of the image outside of the image capturing designation rangecorresponding to that captured image stored in the storing unit, as acurrent remaining capacity. The image capturing apparatus may furtherinclude: an ideal composition detecting unit for detecting an idealcomposition range in which the captured image is ideally composed,wherein the image processing unit may perform an image processing on thecaptured image to make the amount of data of the portion of the imageoutside of either of the image capturing designation range or the idealcomposition range to be less than that of the portion of the imagewithin at least one of the image capturing designation range and theideal composition range, and the storing unit may store informationindicative of the ideal composition range further corresponding to thecaptured image.

The ideal composition range detecting unit may detect the idealcomposition range on the basis of the direction to which a face of aperson in the captured image. Further, the ideal composition rangedetecting unit may detect the ideal composition range in order that abackground area of the captured image in the direction to which the faceof the person in the captured image is facing is wider than anotherbackground area in the opposite direction. Further, the idealcomposition range detecting unit may detect the ideal composition rangeon the basis of the direction to which an object is moving in thecaptured image. Further, the ideal composition range detecting unit maydetect the ideal composition range in order that a background area ofthe captured image in the direction to which the object is moving in thecaptured image is wider than another background area in the oppositedirection. Further, the ideal composition range detecting unit acquiresa position of a vanishing point in the captured image and detects theideal composition range to comprise at least a portion of the imagecapturing designation range and the position of the vanishing point.Here, the ideal composition range detecting unit may detect an areaacquired by expanding the image capturing designation range to comprisethe vanishing point in the direction to the vanishing point as the idealcomposition range in case the vanishing point is outside of the imagecapturing designation range.

Further, the ideal composition range detecting unit may detect a portionof the image capturing designation range to comprise a triangular areaformed by the contour of the object, which has the vanishing point asone of it's apex, as the ideal composition range in case the vanishingpoint is outside of the image capturing designation range. The imagecapturing apparatus may further include: an ideal composition detectingunit for detecting an ideal composition range in which the capturedimage is ideally composed, wherein the image processing unit may performan image processing on the captured image to enhancing the quality ofthe portion of the image within the image capturing designation rangeand the ideal composition range at a level higher than that for theportion of the image outside of either of the image capturingdesignation range or the ideal composition range, and the storing unitmay store information indicative of the ideal composition range furthercorresponding to the captured image.

According to the second aspect of the present invention, an imagecapturing method for capturing an image includes: an image capturingdesignation range acquiring step of acquiring an image capturingdesignation range designated by a user; an image capturing step ofcapturing an image of a range wider than the image capturing designationrange; a storing step of storing information indicative of the imagecapturing designation range corresponding to the captured image; and animage displaying step of displaying at least an image within the imagecapturing designation range out of the captured image by making the userrecognize the image capturing designation range.

According to the third aspect of the present invention, a machinereadable medium storing thereon a computer program, the computer programmaking an image capturing apparatus functions as: an image capturingdesignation range acquiring unit for acquiring an image capturingdesignation range designated by a user; an image capturing unit forcapturing an image of a range wider than the image capturing designationrange; a storing unit for storing information indicative of the imagecapturing designation range corresponding to the captured image; and animage displaying unit for displaying at least an image within the imagecapturing designation range out of the captured image by making the userrecognize the image capturing designation range.

The summary of the invention does not necessarily describe all necessaryfeatures of the present invention. The present invention may also be asub-combination of the features described above. The above and otherfeatures and advantages of the present invention will become moreapparent from the following description of the embodiments taken inconjunction with the accompanying drawings.

According to the present invention, it is possible to provide an imageof a desired composition in case the user is unsatisfied with thecomposition of an image already captured.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a block diagram of an image capturing apparatus 10.

FIG. 2 shows a first example of a captured image 200 according to anembodiment of the present invention.

FIG. 3 shows a second example of the captured image 300 according to anembodiment of the present invention.

FIG. 4 shows an image for illustrating a method for detecting the idealcomposition range on the basis of a facing direction of a person in thecaptured image according to the present embodiment by the idealcomposition range detecting unit 130.

FIG. 5 shows images for illustrating a method for detecting the idealcomposition range on the basis of a moving direction of an object in thecaptured image according to the present embodiment by the idealcomposition detecting unit 130.

FIG. 6 shows an example of a method for detecting an ideal compositionrange on the basis of a vanishing point by using the ideal compositionrange detecting unit 130 according to the present embodiment.

FIG. 7 shows an example of a method for detecting an ideal compositionrange on the basis of a vanishing point by using the ideal compositionrange detecting unit 130 according to the present embodiment.

FIG. 8 shows an example of a process flow of the image capturingapparatus 10 according to the embodiment of the present invention.

FIG. 9 is a block diagram showing an example of a hardware configurationof a computer 1500 according to the embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The invention will now be described based on the preferred embodiments,which do not intend to limit the scope of the present invention, butexemplify the invention. All of the features and the combinationsthereof described in the embodiment are not necessarily essential to theinvention.

FIG. 1 shows a block diagram of an image capturing apparatus 10according to an embodiment of the present invention. The image capturingapparatus 10 is, for example, a digital camera and captures an image.The image capturing apparatus 10 may be a digital still camera forcapturing a still image, and may be a digital video camera for capturinga moving picture.

It is an object of the image capturing apparatus 10 according to thepresent embodiment to let a user perform an editing operation on acaptured image, such as changing the trimming range, by capturing andstoring an image wider than a range designated by the user for capturingthe image in case the user is unsatisfied with the captured image inregard to the composition of the captured image.

The image capturing apparatus 10 includes an image capturing designationrange acquiring unit 100, an image capturing unit 110, a first memory120, an ideal composition range detecting unit 130, an image processingunit 140, a second memory 150, a remaining capacity detecting unit 160,an image removing unit 170 and a displaying unit 180. The imagecapturing designation range acquiring unit 100 acquires an imagecapturing designation range, which is a range in which the userdesignates to capture an image, and outputs information indicating theimage capturing designation range, such as end points, to the imagecapturing unit 110 and the image processing unit 140.

The image capturing unit 110 includes an optical system 112, a CCD 114and an image capturing signal processing unit 116 and captures an image.Here, the image capturing unit 110 captures an image wider than theimage capturing designation range received from the image capturingdesignation range acquiring unit 100. The image capturing unit 110outputs the captured image to the first memory 120. The optical system112 includes a zoom lens, for example, and forms an image of an objecton the light receiving surface of the CCD. Here, it is preferable forthe optical system 112 to be able to control the image capturingmagnification of the zoom lens in order to form an image wider than theimage capturing designation range received from the image capturingdesignation range acquiring unit. The CCD 114 include a plurality oflight receiving elements, and outputs accumulated electrical charges inthe light receiving elements according to the optical image of theobject formed by the optical system 112 as analog electrical signals tothe image capturing signal processing unit 116. The image capturingsignal processing unit 116 decomposes the analog electrical signalsindicating the optical image of the object received from the CCD 114into R, G and B components. The image capturing signal processing unit116 converts the decomposed R, G and B analog signal components intodigital signals, and outputs digital image data indicating the resultingobject image to the first memory 120. The first memory 120 is a volatilememory, such as a DRAM, and stores the image data indicating thecaptured image received from the image capturing signal processing unit116.

The ideal composition range detecting unit 130 detects the idealcomposition range, in which the captured image is ideally composed, fromthe image captured by the image capturing unit 110 and stored in thefirst memory 120. The ideal composition range detecting unit 130 outputsthe captured image and information indicating the ideal compositionrange such as end points to the image processing unit 140. The imageprocessing unit 140 performs an image processing on the captured imagereceived from the ideal composition range detecting unit 130. Here, theimage processing may means image compression according to JPEG (JointPhotographic Experts Group) format, or converting the image signals ofthe captured image into NTSC or PAL video signals. The image processingunit 140 may perform the image processing on the basis of the imagecapturing designation range received from the imate capturingdesignation range acquiring unit 100 and the ideal composition rangereceived from the ideal composition range detecting unit 130.Specifically, the image processing unit 140 may perform an imageprocessing on the captured image in order to make an amount of data ofthe portion of the image outside of either of the image capturingdesignation range or the ideal composition range be less than that ofthe portion of the image within at least one of the image capturingdesignation range and the ideal composition range out of the capturedimage.

Further, the image processing unit 140 may perform AE, AWB, colorprocessing, hue processing, contrast processing or sharpness processing,etc. on the captured image to enhancing the quality of the portion ofthe image within the image capturing designation range and the idealcomposition range. For example, the image processing unit 140 mayperform the image processing to enhance sharpness of the portion of theimage within the image capturing designation range at a higher levelthan that for the portion of the image outside of the image capturingdesignation range. For example, the image processing unit 140 mayperform the image processing to enhance contrast of the portion of theimage within the image capturing designation range at a higher levelthan that for the portion of the image outside of the image capturingdesignation range. For example, the image processing unit 140 mayperform the image processing to enhance hue of the portion of the imagewithin the image capturing designation range at a higher level than thatfor the portion of the image outside of the image capturing designationrange Each AE, AWB, color processing, hue processing, contrastprocessing or sharpness processing is an example of a setup process.

The image processing unit 140 may perform the setup processing on theportions both within and outside of the image capturing designationrange and the ideal composition range. In the related case, the imageprocessing unit 140 may perform the setup processing, such as AE, AWB,color processing, hue processing, contrast processing or sharpnessprocessing, etc., on the portion of the image within the image capturingdesignation range and the ideal composition range at a high level inorder to enhance the quality of the portion of the image within theimage capturing designation range and the ideal composition range betterthan the portion of the image outside of the image capturing designationrange and the ideal composition range.

The image processing unit 140 may enhance the quality of the portion ofthe image within the image capturing designation range and the idealcomposition range better than the portion of the image outside of theimage capturing designation range and the ideal composition range byomitting a part of the setup processing on the portion of the imageoutside of the image capturing designation range and the idealcomposition range. For example, the image processing unit 140 mayperform the setup processing by performing all of the AE, AWB, colorprocessing, hue processing, contrast processing and sharpnessprocessing, etc. on the portion of the image within the image capturingdesignation range and the ideal composition range and performing oneprocessing selected out of the AE, AWB, color processing, hueprocessing, contrast processing and sharpness processing, etc. on theportion of the image outside of the image capturing designation rangeand the ideal composition range. The image processing unit 140 may omitat least some of the image processing included in the setup processingon the portion of the image outside of the image capturing designationrange and the ideal composition range. The image processing unit 140outputs the captured image, on which the image processing is performed,and the information indicating the image capturing designation range andthe ideal composition range to the second memory 150 and the displayingunit 180.

The second memory 150 is a nonvolatile memory, such as a flash memory,and stores the captured image received from the image processing unit140. The second memory 150 further stores information respectivelyindicating the image capturing designation range and the idealcomposition range corresponding to the captured image. The second memory150 is an example of the storing unit of the present invention. Theremaining capacity detecting unit 160 detects the remaining capacity ofthe second memory 150 and outputs information indicating the detectedremaining capacity of the second memory 150 to the image removing unit170 and the displaying unit 180. The remaining capacity detecting unit160 detects the capacity occupied by the portion of the image outside ofeither of the image capturing designation range or the ideal compositionrange corresponding to the captured image stored in the second memory150, and outputs information indicating the detected capacity to thedisplaying unit 180. The image removing unit 170 receives theinformation indicating the remaining capacity of the second memory 150from the remaining capacity detecting unit 160. The image removing unit170 removes the portion of the image outside of either of the imagecapturing designation range or the ideal composition range out of thecaptured image stored in the second memory 150 in case the remainingcapacity of the second memory 150 is less than a predetermined referencevalue.

The displaying unit 180 is an electronic view finder using for examplean LCD panel, and includes an image displaying unit 182 and a remainingcapacity displaying unit 184. The image displaying unit 182 receives thecaptured image and the information indicating the image capturingdesignation range from the image processing unit 140. The imagedisplaying unit 182 displays at least a portion of the captured imagewithin the image capturing designation range by making the userrecognize the image capturing designation range. The remaining capacitydisplaying unit 184 displays a capacity calculated by adding theremaining capacity of the second memory 150 detected by the remainingcapacity detecting unit 160 to the capacity occupied by the portion ofthe image outside of either of the image capturing designation range orthe ideal composition range corresponding to that captured image storedin the second memory 150, as a current remaining capacity of the secondmemory 150.

According to the image capturing apparatus of the present embodiment ofthe present invention, it is possible to capture and store an imagewider than the image capturing designation range designated by the userwhen capturing an image. The user can get another image of a differentcomposition by performing a composition trimming on the captured imagein a wider range than the image capturing designation range when theuser feels that a better image should have been captured if the imagehad been differently composed even after capturing the image.

Further, recently, even though the number of pixels of a CCD used in adigital camera is increased drastically, it is quite general that aresolution of an image used by the user is low in comparison with thenumber of pixels of the CCD due to an image retention environment and/ora reference environment. In other words, there is a case where theincrease in the number of pixels of the CCD compared to the conventionalimage capturing apparatus does not turn to the user's advantage. On theother hand, the image capturing apparatus 10 according to the presentembodiment of the present invention, the image capturing unit 100captures and stores an image wider than the image capturing designantionrange designated by a user. Here, the portion of the captured imagewithin the image capturing designation range has a lower resolution thanthat of the image captured by controlling the zooming ratio for theportion of the image within the image capturing designation range to beformed on the entire light receiving surface of the CCD. However, sincethe number of pixels of the CCD 114 is drastically increased, it ispossible to capture the portion of the image within the image capturingdesignation range with a high enough resolution even when the capturedimage has wider range according to the image capturing apparatus 10. Inother words, it is possible to use the whole pixels of the CCD 114efficiently.

Further, the image capturing apparatus 10 according to the presentembodiment of the present invention is not limited to have the onlystructure shown in FIG. 1, but may have a structure modified by addingvarious changes to the structure shown in FIG. 1. For example, the imagecapturing apparatus 10 may not include the ideal composition rangedetecting unit 130. In this case, each of the image processing unit 140,the second memory 150, the remaining capacity detecting unit 160, theimage removing unit 170 and the remaining capacity displaying unit 184may function without relation to the ideal composition range.Specifically, the image processing unit 140 may perform an imageprocessing on the captured image in order to make an amount of data ofthe portion of the image outside of the image capturing designationrange be less than that of the portion of the image within the imagecapturing designation range out of the captured image. The second memory150 may store the information indicating the image capturing designationrange corresponding to the captured image. The remaining capacitydetecting unit 160 may detect the capacity occupied by the portion ofthe image outside of the image capturing designation range correspondingto the captured image stored in the second memory 150. The imageremoving unit 170 may remove the portion of the image outside of theimage capturing designation range corresponding to the captured imagestored in the second memory 150 in case the remaining capacity of thesecond memory 150 is less than the predetermined reference value.Further, the remaining capacity displaying unit 184 may display acapacity calculated by adding the remaining capacity of the secondmemory 150 detected by the remaining capacity detecting unit 160 to thecapacity occupied by the portion of the image outside of the imagecapturing designation range corresponding to that captured image storedin the second memory 150, as a current remaining capacity of the secondmemory 150.

FIG. 2 shows a first example of a captured image 200 according to anembodiment of the present invention. According to the presentembodiment, the image capturing unit 110 captures the image 200 widerthan the image capturing designation range 210 acquired by the imagecapturing designation range 100. The image processing unit 140 performsan image processing on the basis of the image capturing designationrange on the captured image 200. For example, the image processing unit140 may convert the portion of the image outside of the image capturingdesignation range 210 of the captured image 200 into an image of a lowerresolution than that for the portion of the image within the imagecapturing designation range 210. For another example, the imageprocessing unit 140 may perform an image compression processing on theportion of the image outside of the image capturing designation range210 of the captured image 200 at a higher rate than that for the portionof the image within the image capturing designation range 210.Specifically, the image processing unit 140 may perform the imagecompression processing on the portion of the image outside of the imagecapturing designation range 210 of the captured image 200 with a largerquantization parameter than that for the portion of the image within theimage capturing designation range 210. Further, ‘the higher compressionrate’ means ‘the less data amount of the image after the compressionprocess’.

It is described hereinafter that the image capturing apparatus 10 is animage capturing apparatus for capturing a moving picture, such as adigital video camera. In this case, the image capturing unit 100captures the captured image 200 wider than the image capturingdesignation range as a frame of the moving picture. Further, the imageprocessing unit 140 may decrease the amount of data of the portion ofthe image outside of the image capturing designation range 210 byremoving more high frequency components from the image signalsindicating the portion of the image outside of the image capturingdesingantion range 210 than from those indicating the portion of theimage within the image capturing designation range 210 out of the imagesignals indicating the captured image 200.

According to the image capturing apparatus 10 of the present embodiment,it is possible to use a limited storing capacity of the second memory150 because it is possible to make the portion of the image outside ofthe image capturing designation range be an image of relatively smallamount of data by controlling the resolution and/or the compressionrate. Further, in case the image capturing unit 10 captures and stores astill image, there are cases where the image quality varies across theimage capturing designation range due to the resolution and/or thecompression rate. However, in general, when the image capturingapparatus 10 captures a moving picture, even though there aredifferences in the quantity of removing the high frequency componentsacross the image capturing designation range, it is difficult for theuser to recognize the difference during watching the captured movingpicture, so, generally, it does not make a big problem. In other words,in this case, it is possible to store a moving picture by using thestoring capacity of the second memory efficently without deterioratingthe quality of the portion of the image outside of the image capturingdesignation range.

FIG. 3 shows a second example of the captured image 300 according to anembodiment of the present invention. According to the presentembodiment, the image capturing unit 110 captures an image wider thanthe image capturing designation range 310 acquired by the imagecapturing designation range acquiring unit 100. The ideal compositionrange detecting unit 130 detects an ideal composition range 320, inwhich the captured image is ideally composed, out of the captured image300. For example, the ideal composition range detecting unit 130 firstextracts a contour of an object in the captured image 300 by performingan edge extraction processing on the captured image 300. Then, the idealcomposition range detecting unit 130 detects a main object out of theobject(s) from which the contour(s) is(are) extracted. For example, theideal composition range detecting unit 130 recognizes an object ofrelatively more skin color than other object(s) as a human and detectsthat object as the main object by detecting color distribution(s) of theobject(s) of which contour(s) is(are) extracted in the captured image300. Then, the ideal composition range detecting unit 130 determines anddetects an ideal composition range 320 in which the detected main objectis of an optimal position and size. Here, the ideal composition rangedetecting unit 130 may detect the ideal composition range on the basisof the position of the main object according to a generally acceptedmethod for determining a well-composed image, such as a tripartitemethod. Further, for example, the ideal composition range detecting unit130 may detects the ideal composition range 320 on the basis ofinformation indicating the ideal composition including the position andsize of the main object and the ratio of the size of the main object tothe size of the ideal composition range 320.

The image processing unit 140 performs on the basis of the imagecapturing designation range 310 and the ideal composition range 320 onthe captured image 300. Here, the image processing unit 140 may performthe image processing, such as the resolution and/or compression ratecontrolling, high frequency component removing and image qualityenhancing, etc., on the portion of the image within at least one of theimage capturing designation range and the ideal composition range andthe other portion of the image outside of either of the image capturingdesignation range or the ideal composition range, respectively. Forexample, the image processing unit 140 may convert the portion of theimage outside of either of the image capturing designation range 310 orthe ideal composition range 320 into an image of lower resolution thanthat of the portion of the image within at least one of the imagecapturing designation range 310 and the ideal composition range 320.

According to the image capturing apparatus of the present embodiment, itis possible to store an image of a better quality in resulution and/orcompression rate in the ideal composition range as well as in the imagecapturing desingation range designated by the user. According to this,the user can have a high quality image by performing an editingoperation, such as a trimming operation, on the captured image when theuser feels that the user should have captured a better image if theimage was in another composition even after the image has been captured.

FIG. 4 shows an image for illustrating a method for detecting the idealcomposition range on the basis of a facing direction of a person in thecaptured image according to the present embodiment by the idealcomposition range detecting unit 130. According to the presentembodiment, the object in the captured image may be a human, and theideal composition range detecting unit 130 may identify the facingdirection of the person in the captured image. The ideal compositionrange detecting unit 130 may identify the facing direction of the faceof the person in the captured image. The ideal composition rangedetecting unit 130 extracts the image portion of the person in thecaptured image. The ideal composition range detecting unit 130 mayextract the vertically symmetrical area of the image of the person byperforming, for example, a blurring processing and the matchingprocessing on the extracted image of the person. The ideal compositionrange detecting unit 130 may identify the facing direction of the personin the captured image on the basis of the concentration of the positionof the portion other than the front face of the person in the capturedimage.

The ideal composition range detecting unit 130 extracts the area of theface of the person out of each extracted object by the image processing,such as determining an area matched with a template facial image ofvarious positions, sizes or rotated angles, etc. or extracting skincolor. The ideal composition range detecting unit 130 may extract avertically symmetrical area of the face of the person by performing, forexample, the blurring processing on the contour of the extracted face ofthe person and then the matching processing. The ideal composition rangedetecting unit 130 may identify the facing direction of the face of theperson in the captured image on the basis of the area occupied by theportions other than the front face of the person. For example, in casethe vertically symmetrical area of the face of the person occupies theright side of the face, the facing direction of the face of the personmay be identified as the right direction.

The ideal composition range detecting unit 130 may store data of afacial image of a person in advance. The ideal composition rangedetecting unit 130 performs a pattern matching on a plurality ofportions of facial images of the extracted images of the persons in thecaptured image with the facial image data stored in advance in the idealcomposition range detecting unit 130. The ideal composition rangedetecting unit 130 extracts a portion of a face of a person. The idealcomposition range detecting unit 130 identifies the facing direction ofthe face of the person in the captured image on the basis of theplurality of portions in the extracted facial area. The idealcomposition range detecting unit 130 may identify the facing directionof the person 432 in the captured image 400 as the direction of the eyes434, the right direction, in case, for example, the eyes 434 extractedfrom the face of the person 432 are concentrated in the right side ofthe face. The ideal composition range detecting unit 130 may identifythe facing direction of the face of the person in the captured image asthe direction of the end of the nose 436 by detecting the direction ofthe end of the nose 436 in case the nose 436 is extracted from the faceof the person 432, for another example. In the above example, since theend of the nose 436 of the person 432 faces to the right direction inthe captured image 400, the facing direction of the person 432 in thecaptured image 400 may be identified as the right direction.

The ideal composition range detecting unit 130 may identify a facingdirection of the body of the person as a direction perpendicular to theshoulder of the person. It is also possible to determine the facingdirection of the body is the facing direction of the person in thecaptured image. The direction of the person can be identified asdescribed hereinafter, for example. First, the ideal composition rangedetecting unit 130 extracts the face of the person. Then, the idealcomposition range detecting unit 130 recognizes the shoulder positionedbelow the extracted face of the person. Then, the ideal compositionrange detecting unit 130 identifies the facing direction of the shoulderon the basis of, for example, the front-rear relationship of the upperarms of both arms in the captured image. For example, in case the arearatio of the right arm to the are of the captured image is higher thanthat of the left arm, it is possible to determine the right arm is infront of the left arm. In other words, it is possible to determine thatthe facing direction of the shoulder in the captured image is the leftdirection. In this case, the facing direction of the body of the personin the captured image is the right direction perpendicular to thedirection of the shoulder.

Here, in case the facing directions of the face and the body of theperson identified by the ideal composition range detecting unit 130 arethe same, the facing direction of the person in the captured image isidentified as the identified direction of the face or the body of theperson. On the other hand, in case the facing direction of the face ofthe person identified by the ideal composition range detecting unit 130and the facing direction of the body of the person identified by theideal composition range detecting unit 130 on the basis of the facingdirection of the shoulder of the person are not the same, the facingdirection of the person in the captured image may be identified asdescribed hereinafter. For example, even though the identified directionof the face is the front direction of the captured image, in case thefacing direction of the body in the captured image is not the frontdirection, the facing direction of the person in the captured image isidentified as the facting direction of the body of the person. On theother hand, even though the identified direction of the face is not inthe front direction of the captured image, in case the identifieddirection of the body is in the front direction of the captured image,the facing direction of the person in the captured image is identifiedas the facting direction of the face of the person.

The ideal composition range detecting unit 130 may determine acomposition in which a blank space is located in the facing direction ofthe face of the extracted person as the ideal composition. The idealcomposition range detecting unit 130 may detect the ideal compositionrange 420 indicating the ideal composition. By this, the storing unit ofthe image capturing apparatus 10 may store the information indicatingthe image capturing desigantion range 410 and the ideal compositionrange 420 for the captured image 400.

FIG. 5 shows images for illustrating a method for detecting the idealcomposition range on the basis of a moving direction of an object in thecaptured image according to the present embodiment by the idealcomposition detecting unit 130. The ideal composition detecting unit 130detects positions of the same object in frame images 520 and 524captured at times t500 and t504, respectively. Specifically, the idealcomposition range detecting unit 130 extracts contours of the object 580in the frame images 520 and 524 captured at the times t500 and t504,respectively, by the image processing, such as edge extraction, etc. Theideal composition range detecting unit 130 detects positions of the sameobject 580 in the frame images at times t500 and t504 by performing, forexample, the pattern matching between the extracted contours.

The ideal composition range detecting unit 130 acquires a movingdirection, a direction in which the object 580 is moving, on the basisof the positions of the object 580 in the frames at times t500 and t504and the frame period. Then, the ideal composition range detecting unit130 compares the size of a background area in the moving direction ofthe object 580 in the image capturing designation range 542 of thecaptured image 540 with the size of the background area in the reversedirection of the moving direction of the object 580 in the capturedimage 540. The ideal composition range detecting unit 130 detects theideal composition range 544 in order that the background area in themoving direction of the object 580 in the captured image 540 is widerthan the background area in the reverse direction of the movingdirection of the object 580 in the captured image 540.

FIG. 6 shows an example of a method for detecting an ideal compositionrange on the basis of a vanishing point by using the ideal compositionrange detecting unit 130 according to the present embodiment. The idealcomposition range detecting unit 130 acquires a direction component of acaptured image 600 by performing an image processing such as an edgeextraction processing, a line thinning processing, etc. on the capturedimage 600. Further, the ideal composition range detecting unit 130acquires a vanishing point 630 on which the acquired direction componentis focused. Then, the ideal composition range detecting unit 130 detectsan ideal composition range 620 including at least a part of an imagecapturing designation range 610 and the position of the acquiredvanishing point 630. The ideal composition range detecting unit 130 maydetect an area acquired by expanding the image capturing designationrange 610 to comprise the vanishing point 630 in the direction to thevanishing point 630 as the ideal composition range 620 in case thevanishing point 630 is outside of the image capturing designation range610. In a related case, the ideal composition range detecting unit 130may detect the ideal composition range 620 by expanding the imagecapturing designation range 610 so that the aspect ratio of the idealcomposition range 620 can be substantially the same as that of the imagecapturing designation range 610. Further, the ideal composition rangedetecting unit 130 may expand or reduce the area of the image capturingdesignation range 610 by lengthening or shortening the longer side orthe shorter side of the image capturing designation range 610 to detectthe ideal composition range 620 comprising the vanishing point. Forexample, the ideal composition range detecting unit 130 may detect theideal composition range 620 comprising the vanishing point by shorteningthe longer side and lengthening the shorter side of the image capturingdesignation range 610.

FIG. 7 shows an example of a method for detecting an ideal compositionrange on the basis of a vanishing point by using the ideal compositionrange detecting unit 130 according to the present embodiment. The idealcomposition range detecting unit 130 acquires a direction component of acaptured image 800 by performing an image processing such as an edgeextraction processing, a line thinning processing, etc. on the capturedimage 800. Further, the ideal composition range detecting unit 130acquires a vanishing point 830 on which the acquired direction componentis focused. The ideal composition range detecting unit 130 detects atriangular partial area 840 formed by the contour of an object, whichhas the vanishing point as one of it's apex, in case the vanishing point830 is outside of the image capturing designation range 810. Further,the ideal composition range detecting unit 130 detects a portion of theimage capturing designation range 810 to comprise the detectedtriangular partial area 840. In addition, the number of the triangularpartial area(s) 840 formed by formed by the contour of an object may beone or a plural number for the captured image 800.

FIG. 8 shows an example of a process flow of the image capturingapparatus 10 according to the embodiment of the present invention.First, the image capturing designation range acquiring unit 100 acquiresan image capturing designation range, which is a range in which the userdesignates to capture an image, in case of being instructed of capturingthe image, for example, by pressing a release button (S1000). Forexample, the image capturing designation range acquiring unit 100 mayacquire a range displayed on an electronic view finder which is thedisplaying unit 180 by the user's manual adjustment of a zooming ratioas the image capturing designation range. Then, the image capturing unit110 captures an image of a range wider than the image capturingdesignation range (S1010).

Then, the ideal composition range detecting unit 130 detects an idealcomposition range in which the captured image is ideally composed fromthe captured image (S1020). Then, the image processing unit 140 performsan image processing on the captured image on the basis of the imagecapturing designation range and the ideal composition range (S1030).Specifically, the image processing unit 140 may perform an imageprocessing on the captured image to make the amount of data of a portionof the captured image outside of either of the image capturingdesignation range or the ideal composition range to be less than that ofthe portion of the image within at least one of the image capturingdesignation range and the ideal composition range. Here, the imageprocessing unit 140 may confirm whether the image processing is to beperformed to the user before performing the image processing on thecaptured image. Further, the image processing unit 140 may confirmwhether or not the image processing is to be performed to the user onlyif it is previously instructed of requiring the user's confirmation inwhich case such the image capturing apparatus 10 is validly set to aconfirmation mode. Furthermore, in case of requiring the user'sconfirmation on whether or not the image processing is to be performed,the image processing unit 140 may display images of a portion of theimage which is more largely than the other part before and after theprocessing by using the displaying unit 182 so that the user can comparethem, for example, by dividing a screen to display them simultaneouslyor making them displayed switchably. Then, the image displaying unit 140displays at least an image within the image capturing designation rangeout of the captured image by making the user recognize the imagecapturing designation range (S1040). For example, the image displayingunit 182 may display only the image capturing designation range out ofthe captured image, or an image of a range wider than the imagecapturing designation range with a frame showing the image capturingdesignation range overlapped. Then, the second memory 150 storesinformation respectively indicating the image capturing designationrange and the ideal composition range corresponding to the capturedimage (S1050).

Then, the remaining capacity detecting unit 160 detects the remainingcapacity of the second memory 150 (S1060). Here, the remaining capacitymay be a byte unit value of the data amount on which additional writingis possible or a value showing the number of captured images which canbe additionally written for resolution and color depth of a currentimage capturing mode of the image capturing apparatus 10, that is, avalue obtained by dividing the data amount on which additional writingis possible by an average data amount per image for the image capturingmode. Then, the image removing unit 170 judges whether or not theremaining capacity of the second memory 150 is less than a predeterminedreference value (S1070). Here, the predetermined reference value may bea byte unit value showing the data amount or the number of imagescaptured by using the current image capturing mode of the imagecapturing apparatus 10.

Further, in case it is judged that the remaining capacity of the secondmemory 150 is less than a predetermined reference value (S1070: Yes),the image removing unit 170 removes the portion of the image outside ofeither of the image capturing designation range or the ideal compositionrange out of the captured image stored in the second memory 150 (S1080).Thus, even in case an image outside of either of the image capturingdesignation range and the ideal composition range is stored, it ispossible for the user to use efficiently memory capacity of the secondmemory 150 without reducing the number of images which can be capturedby the image capturing apparatus 10. Further, the image removing unit170 may remove a portion of the image which is inside of the idealcomposition range but outside of the image capturing designation rangeas well as a portion of the image outside of the image capturingdesignation range and the ideal composition range. In addition, in caseof performing a processing of removing a partial image included in thecaptured image as above, the image capturing apparatus 10 may confirmwhether or not the processing is to be performed to the user as similarwith S1030 in which the image processing unit 140 performs an imageprocessing. Further, in case of performing the processing automatically,the image capturing apparatus 10 may inform the user of that, forexample, by displaying on the displaying unit 182.

Then, the remaining capacity displaying unit 184 displays a capacitycalculated by adding the remaining capacity of the second memory 150detected by the remaining capacity detecting unit 160 to the capacityoccupied by the portion of the image outside of the image capturingdesignation range and the ideal composition range corresponding to thatcaptured image stored in the second memory 150, as a remaining capacityof the second memory 150 (S1090). Here, the remaining capacitydisplaying unit 184 may display a byte unit value of the data amount onwhich additional writing is possible or a value showing the number ofcaptured images which can be additionally written for a current imagecapturing mode of the image capturing apparatus 10 as the remainingcapacity of the second memory 150. Further, the remaining capacitydisplaying unit 184 may display a capacity calculated by adding acapacity occupied by the portion which is inside the ideal compositionrange but outside of the image capturing designation range as well asthe portion of the image outside of the image capturing designationrange and the ideal composition range to the detected remainingcapacity.

In case of displaying the capacity detected by the remaining capacitydetecting unit 160 without any retouch, a remaining capacity smallerthan that in case of storing images inside the image capturingdesignation range is displayed since image(s) outside of the imagecapturing designation range is(are) stored in the second memory 150.However, since the image removing unit 170 removes image(s) outside ofthe image capturing designation range when the remaining memory of thesecond memory 150 is reduced, the image capturing apparatus 10 shoulddisplay a remaining capacity substantially the same as that in case ofstoring only images inside of the image capturing designation range.According to the image capturing apparatus 10 of the embodiment of thepresent invention, the remaining capacity displaying unit 184 displays aremaining capacity calculated by adding a capacity occupied by an imageoutside of the image capturing resignation region to the detectedremaining capacity and thus it is possible to display a remainingcapacity substantially the same as that in case of storing only imagesinside of the image capturing designation range even in case of storingimages outside of the image capturing designation range.

FIG. 9 is a block diagram showing an example of a hardware configurationof a computer 1500 according to the embodiment of the present invention.The computer 1500 according to the embodiment of the present inventionincludes a CPU peripheral part comprising a CPU 1505, a RAM 1520, agraphic controller 1575, and a displaying apparatus 1580 which areconnected with each other by a host controller 1582, an input and outputpart comprising a communication interface 1530 connected to the hostcontroller 1582 by an input and output controller 1584 (“I/O controller1584”), a hard disk drive 1540, and a CD-ROM drive 1560, and a legacyinput and output part comprising a ROM 1510 connected to the I/Ocontroller 1584, a flexible disk drive 1550, and an input and outputchip 1570 (“I/O chip 1570”).

The host controller 1582 connects the RAM 1520 with the CPU 1505 havingaccess to the RAM 1520 at a high transmission rate and the graphiccontroller 1575. The CPU 1505 operates and performs control of each parton the basis of programs stored in the ROM 1510 and the RAM 1520. Thegraphic controller 1575 acquires image data which is generated on aframe buffer provided in the RAM 1520 by the CPU 1505, etc. and displaysit on the displaying apparatus 1580. Instead, the graphic controller1575 may include a frame buffer storing an image data generated by theCPU 1505, etc. therein.

The I/O controller 1584 connects the host controller 1582 with the harddisk drive 1540, which is a relatively high-speed I/O apparatus, thecommunication interface 1530, and the CD-ROM drive 1560. The hard diskdrive 1540 stores a program and data to be used by the CPU 1505 in thecomputer 1500. The communication interface 1530 communicates with theimage capturing apparatus 10 via a network and provides the imagecapturing apparatus 10 with a program and data. The CD-ROM drive 1560retrieves a program or data from a CD-ROM 1595 and provides the harddisk drive 1540 and the communication interface 1530 with it via the RAM1520.

Further, the I/O controller 1584 is connected with the ROM 1510 and arelatively low-speed I/O apparatus such as the flexible disk drive 1550and the I/O chip 1570. The ROM 1510 stores a boot program executed bythe CPU 1500 when the computer 1500 starts to operate, a programdepending on the hardware of the computer 1500, and the like. Theflexible disk drive 1550 retrieves a program or data from a flexibledisk 1590 and provides the hard disk drive 1540 and the communicationinterface 1530 with it via the RAM 1520. The I/O chip 1570 connects theflexible disk drive 1550 with various I/O apparatus via a parallel port,a serial port, a keyboard port, a mouse port, and the like.

The program provided to the communication interface 1530 via the RAM1520 is stored in a recording medium such as the flexible disk 1590, theCD-ROM 1595, or an IC-card and provided by a user. The program isretrieved from the recording medium, provided to the communicationinterface 1530 via the RAM 1520, and sent to the image capturingapparatus 10 via the network. The program sent to the image capturingapparatus 10 is installed on the image capturing apparatus 10 to beexecuted. The program installed and executed on the image capturingapparatus 10 makes the image capturing apparatus 10 operate as the imagecapturing apparatus 10 described with respect to FIGS. 1 to 8.

The program described above may be stored in a recording medium of theoutside. An optical recoding medium such as a DVD, a PD, etc., amagneto-optical recording medium such as an MD, a tape medium, and asemiconductor memory such as an IC card can be used as the recodingmedium in addition to the flexible disk 1590 and the CD-ROM 1595.Further, a storing apparatus such as a hard disk or a RAM provided in aserver system connected with a dedicated communication network andinternet may be used as the recording medium and may provide thecomputer 1500 with the program through the network.

Although the present invention has been described by way of exemplaryembodiments, it should be understood that those skilled in the art mightmake many changes and substitutions without departing from the spiritand the scope of the present invention which is defined only by theappended claims.

1. An image capturing apparatus for capturing an image comprising: animage capturing designation range acquiring unit for acquiring an imagecapturing designation range designated by a user; an image capturingunit for capturing an image including the image capturing designationrange and a range wider than the image capturing designation range; astoring unit for storing information indicative of the image capturingdesignation range at a first data amount and storing informationindicative of the image captured by the image capturing unit outside theimage capturing designation range at least partly at a second dataamount which is less than the first data amount; an image displayingunit for displaying at least an image within the image capturingdesignation range out of the captured image by making the user recognizethe image capturing designation range, an image processing unit forperforming an image processing on the captured image in order to make anamount of data of the portion of the image outside of the imagecapturing designation range be less than that of the portion of theimage within the image capturing designation range out of the capturedimage, and an ideal composition detecting unit for detecting an idealcomposition range in which the captured image is ideally composed, andwhich includes an area that is not a part of said image capturingdesignation range, wherein said image processing unit performs an imageprocessing on the captured image to make the amount of data of theportion of the image outside of either of the image capturingdesignation range or the ideal composition range to be less than that ofthe portion of the image within at least one of the image capturingdesignation range and the ideal composition range, and said storing unitstores information indicative of the ideal composition range furthercorresponding to the captured image, and wherein said ideal compositionrange detecting unit detects the ideal composition range on the basis ofthe direction to which an object is moving in the captured image.